Math Search for the Masses: Multimodal Search Interfaces and Appearance-Based Retrieval

We summarize math search engines and search interfaces produced by the Document and Pattern Recognition Lab in recent years, and in particular the min math search interface and the Tangent search engine. Source code for both systems are publicly available. "The Masses" refers to our emphasis on creating systems for mathematical non-experts, who may be looking to define unfamiliar notation, or browse documents based on the visual appearance of formulae rather than their mathematical semantics.Comment: Paper for Invited Talk at 2015 Conference on Intelligent Computer Mathematics (July, Washington DC

Moving Away from Programming and Towards Computer Science in the CS First Year

After completing a pilot study using the Python programming language to transition to Java within our first-year introductory programming sequence, our department opted to make a more radical change. We assert that our students are better served in their first year of study by a focus on problems in computer science and their solutions, rather than programming. Our new introductory sequence emphasizes algorithm development and analysis, object-oriented design, and testing. As in our pilot, programming is first done in Python, switching to Java when object-oriented design and static typing become advantageous. Students reported liking the problem focus of the courses, while the distribution of grades remained similar to those in previous years. As a result, our department will be discontinuing our earlier introductory sequence, and offering the new problem-based one to all the groups of students our department services beginning in Fall 2010

Identifying Layout Classes for Mathematical Symbols Using Layout Context

We describe a symbol classification technique for identifying the expected locations of neighboring symbols in mathematical expressions. We use the seven symbol layout classes of the DRACULAE math notation parser (Zanibbi, Blostein, and Cordy, 2002) to represent expected locations for neighboring symbols: Ascender, Descender, Centered, Open Bracket, Non-Script, Variable Range (e.g. integrals) and Square Root. A new feature based on shape contexts (Belongie et al., 2002) named layout context is used to describe the arrangement of neighboring symbol bounding boxes relative to a reference symbol, and the nearest neighbor rule is used for classification. 1917 mathematical symbols from the University of Washington III document database are used in our experiments. Using a leave-one-out estimate, our best classification rate reaches nearly 80%. In our experiments, we find that the size of the symbol neighborhood, and number and arrangement of key points representing a symbol affect performance significantly

Video CAPTCHAs: Usability vs. Security

A Completely Automated Public Turing test to tell Computer and Humans Apart (CAPTCHA) is a variation of the Turing test, in which a challenge is used to distinguish humans from computers (‘bots’) on the internet. They are commonly used to prevent the abuse of online services; for example, malicious users have written automated programs that sign up for thousands of free email accounts and send SPAM messages. A number of hard artificial intelligence problems, including natural language processing, speech recognition, character recognition, and image understanding, have been used as the basis for these challenges on the expectation that humans will outperform bots. The most common type of CAPTCHA requires a user to transcribe distorted characters displayed within a noisy image. Unfortunately, many users find CAPTCHAs based on character-recognition frustrating and attack success rates as high as 60% have been reported for Microsoft’s Hotmail CAPTCHA [8].To address these problems, we present a first attempt at using content-based video labeling (‘tagging’) as a the basis for a CAPTCHA

Decision-Based Specification and Comparison of Table Recognition Algorithms

The vast majority of algorithms in the table recognition literature are specified informally as a sequence of operations [7]. This has the undesirable side effects that models of table structure are implicit, defined generatively by the sequence of operations, and that the effects of intermediate decisions are often lost as usually a single interpretation is modified in-place. We wished to compare the Handley [2] and Hu et al. [4]. table structure recognition algorithms and the complete set of table cell hypotheses they each generated, including any rejected in the final result. Rebuilding the systems using procedural code that transformed data structures for interpretations in-place would not have achieved this goal. Initially we translated the strategies to a formal model-based (specifically grammarbased) framework. A well designed model-driven system (such as DMOS by Couasnon ¨ [1]) makes it easier to observe and record decision making, and can be programmed succinctly by a model specification. However, we found mapping the sequence of operations in the strategies to a model based description was difficult, and the formal system required frequent and substantial reconfiguration in order to incorporate unanticipated requirements. We then considered an intermediate level of formalization. By using a small set of basic graph-based operations we could define recognition algorithms as a series of decisions, where the alternatives for each decision were model operations of a specified type (e.g. classifying table cells as header cells or data cells). This made the model operations considered and applied at each decision point explicit, permitted dependencies between logical types to be automatically recovered, and allowed the complete history of hypothesis creation, rejection, and reinstatement to be automatically captured. The resulting formalization is the Recognition Strategy Language (RSL)